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August, 2004 

Each year, South Carolina students participate in testing as part of the South Carolina assessment 
program. Students in grades 3 through 8 take the Palmetto Achievement Challenge Tests (PACT) in 
English/Language Arts and Mathematics. Students in grade 10 take the High School Assessment 
Program (PACT) in English/Language Arts and mathematics. These tests serve as an important measure 
of student achievement for the state’s accountability system. Results from these assessments are used to 
make state-level decisions concerning education, to meet Adequate Yearly Progress (AYP) reporting 
requirements of the No Child Left Behind Act (NCLB), and to inform schools and school districts of their 
performance. The South Carolina Department of Education has developed scales that are used to assign 
students to one of four performance levels on these tests. 

Many students who attend school in South Carolina also take tests developed in cooperation with the 
Northwest Evaluation Association (NWEA). These tests report student performance on a single, cross- 
grade scale, which NWEA calls the RIT scale. This scale was developed using Rasch scaling 
methodologies. RIT-based tests are used to inform a variety of educational decisions at the district, 
school, and classroom level. They are also used to monitor academic growth of students and cohorts. 
Districts choose whether to include these assessments in their local assessment programs. They are not 
state mandated. 

In order to use the two testing systems to support each other, an alignment of the scores from the state and 
RIT-based tests is as important as the curriculum alignment. NWEA has conducted two prior studies to 
investigate the alignment of cut scores between the spring 2002 administrations of the PACT and NWEA 
tests (Cronin, 2003). We have also just published a study investigating the alignment between the High 
School Assessment Program (PACT) tests and the NWEA scale (Cronin, 2004). 

The current study of the PACT was undertaken in an effort to monitor the accuracy and stability of 
NWEA estimated cut scores relative to these tests. In addition, we have expanded the scope of this study 
to include estimation of cut scores in language usage that would correspond to each proficiency level on 
the English/Language Arts portion of the PACT. 

The primary questions addressed in this study are: 

■ What RIT scores correspond to various performance levels on the PACT tests? 

■ How do these RIT scores differ from the 2002 estimates of performance levels? 

■ How well can performance on the South Carolina assessments be predicted from RIT scores 
when NWEA assessments are administered in the same time frame? 

Method 

Over 22,000 test records of students from the Horry County, Richland 2, and Charleston County school 
systems were included in this study. Student records were included when a student had both a valid 
NWEA scale score and a valid PACT score in the equivalent subject. 

The methodology used to complete this validation study was identical to that used in prior studies to 
establish alignment of the RIT scale with state tests (Kingsbury et al, 2004) and in the two prior studies of 




alignment with the PACT scale. To conserve space, we refer readers to the more recent study (Cronin, 
2003), which is available on our website, for more detail about the methods we use to conduct scale 
alignment studies. 

Results 

Descriptive Statistics 

Table 1 reviews descriptive statistics for the PACT and NWEA assessments. The median RIT scores for 
this sample in reading and language usage are near the median for the NWEA norm population. The 
median RIT score in mathematics, however, is considerably above the median for the NWEA norm 
population. Eor example, the grade 6 mathematics medians for the study group exceed the national 
median by .57 standard deviations. 

Normal distributions around a nationally-normed mean are desirable but not necessarily essential when 
conducting alignment studies. It is more important that the sample provide reasonable numbers of 
students who perform at all levels on the test scales than normal distribution so that the statistical methods 
applied have an adequately large sample to derive good estimates of performance levels that are at the 
higher and lower ends of a test scale. In this case we had reasonably large representations of students 
who performed at all performance levels. 



Table 1 — Means, Standard Deviations, and Medians for PACT and NWEA assessments 



PACT English/Language Arts 


Grade 


3 


4 


5 


6 


7 


8 


N 


3517 


3612 


3680 


4023 


3996 


3714 


Mean 


316.13 


408.33 


506.69 


605.95 


706.62 


807.42 


Median 


317 


409 


507 


607 


706 


807 


Std. Dev. 


14.90 


12.72 


13.12 


16.07 


13.33 


12.26 


1 


NWEA Reading 


Grade 


3 


4 


5 


6 


7 


8 


N 


3517 


3612 


3680 


4023 


3996 


3714 


Mean 


199.99 


207.16 


213.00 


216.61 


220.19 


223.52 


Median 


201 


208 


214 


217 


221 


225 


Std. Dev. 


12.37 


1 1.43 


11.17 


11.69 


11.61 


1 1.40 


NWEA Language Usage 


Grade 


3 


4 


5 


6 


7 


8 


N 


2433 


2456 


2493 


2585 


2515 


2273 


Mean 


202.65 


208.67 


214.65 


217.20 


219.33 


222.29 


Median 


203 


209 


215 


218 


220 


223 


Std. Dev. 


11.99 


1 1.22 


10.33 


10.00 


10.48 


10.09 


PACT Mathematics 


Grade 


3 


4 


5 


6 


7 


8 


N 


3481 


3569 


3654 


3992 


3945 


3681 


Mean 


313.02 


414.70 


506.65 


615.58 


713.66 


810.53 


Median 


314 


415.00 


507 


618 


714 


811 


Std. Dev. 


12.43 


14.67 


13.13 


14.95 


15.41 


13.88 


NWEA Mathematics 


Grade 


3 


4 


5 


6 


7 


8 


N 


3481 


3569 


3654 


3992 


3945 


3681 


Mean 


207.07 


216.99 


225.72 


229.79 


234.83 


239.56 


Median 


207 


217 


227 


230 


236 


240 


Std. Dev. 


11.69 


12.62 


12.89 


13.59 


14.09 


14.53 




Pearson correlations 



Table 2 shows the results of this analysis for each grade. Concurrent validity was tested by examining 
same subject Pearson correlations between the NWEA and PACT. Same subject correlations were 
generally high, ranging from .73 to .85, numbers that suggest the tests were generally measuring the same 
constructs. Discriminant validity was tested by examining same subject Pearson correlations next to 
correlations for the alternate subject (math against reading and language usage). In all cases the same 
subject correlations were higher than correlations against the alternate subject. 



Table 2 - Inter-test Correlations for PACT and NWEA assessments by Subject 



Grade 3 




PACT 


NWEA 




ELA 


Mathematics 


Reading 


Language 

Usage 


Math 


PACT ELA 


1 


.63 


.76 


.79 


.65 


PACT Mathematics 


.65 


1 


.55 


.66 


.76 


Grade 4 




PACT 


NWEA 




ELA 


Mathematics 


Reading 


Language 

Usage 


Math 


PACT ELA 


1 


.74 


.79 


.80 


.65 


PACT Mathematics 


.74 


1 


.55 


.66 


.84 


Grade 5 




PACT 


NWEA 




ELA 


Mathematics 


Reading 


Language 

Usage 


Math 


PACT ELA 


1 


.76 


.78 


.79 


.72 


PACT Mathematics 


.77 


1 


.72 


.75 


.84 


Grade 6 




PACT 


NWEA 




ELA 


Mathematics 


Reading 


Language 

Usage 


Math 


PACT ELA 


1 


.74 


.77 


.79 


.69 


PACT Mathematics 


.72 


1 


.68 


.71 


.84 


Grade 7 




PACT 


NWEA 




ELA 


Mathematics 


Reading 


Language 

Usage 


Math 


PACT ELA 


1 


.70 


.78 


.78 


.55 


PACT Mathematics 


.57 


1 


.70 


.71 


.85 


Grade 8 




PACT 


NWEA 




ELA 


Mathematics 


Reading 


Language 

Usage 


Math 


PACT ELA 


1 


.73 


.76 


.73 


36 


PACT Mathematics 


.37 


1 


.68 


.67 


.85 



Same subject correlations are shaded 





Linking PACT performance level cut scores to the RIT scale 



The primary purpose of this study was to estimate the RIT scale scores that most closely correspond to the 
cut scores for different performance levels on the PACT. This information allows schools to identify 
students who may need additional support to reach state standards. It can also help schools identify 
students who are performing well enough that they are ready to tackle work beyond what the state 
standards require. 

Table 3 shows several estimations of the Spring 2003 RIT score that correspond to the cut scores for the 
various performance levels on the PACT scales. As a rule the three methodologies came to similar 
estimates of cut scores for each of the performance levels, although the Rasch SOS methodology did 
produce somewhat higher estimates of the RIT score required to meet the basic standard at some grades. 



Table 3 - Estimated points on the RIT scale equating to the minimum scores (rounded) for 
performance levels on the PACT 





Grade 3 




Linear Regression 


Second-order Regression 


Rasch Status-on-Standard 




BB 


B 


P 


A 


BB 


B 


P 


A 


BB 


B 


P 


A 


Reading 


<179 


179 


193 


216 


<180 


180 


196 


217 


<182 


182 


196 


211 


Language Usage 


<182 


182 


196 


217 


<182 


182 


198 


218 


<186 


186 


198 


213 


Mathematics 


<190 


190 


212 


226 


<186 


186 


208 


220 


<193 


193 


210 


220 




Grade 4 




Linear Regression 


Second-order Regression 


Rasch Status-on-Standard 




BB 


B 


P 


A 


BB 


B 


P 


A 


BB 


B 


P 


A 


Reading 


<192 


192 


209 


232 


<190 


190 


208 


229 


<194 


194 


209 


226 


Language Usage 


<195 


195 


211 


232 


<194 


194 


210 


229 


<197 


197 


210 


226 


Mathematics 


<201 


201 


219 


231 


<200 


200 


218 


228 


<202 


202 


219 


228 




Grade 5 




Linear Regression 


Second-order Regression 


Rasch Status-on-Standard 




BB 


B 


P 


A 


BB 


B 


P 


A 


BB 


B 


P 


A 


Reading 


<200 


200 


218 


240 


<199 


199 


216 


236 


<202 


202 


217 


232 


Language Usage 


<203 


203 


219 


239 


<204 


204 


221 


238 


<204 


204 


218 


231 


Mathematics 


<211 


211 


228 


240 


<211 


211 


230 


239 


<212 


212 


228 


236 




Grade 6 




Linear Regression 


Second-order Regression 


Rasch Status-on-Standard 




BB 


B 


P 


A 


BB 


B 


P 


A 


BB 


B 


P 


A 


Reading 


<208 


208 


222 


240 


<206 


206 


220 


236 


<210 


210 


220 


233 


Language Usage 


<209 


209 


222 


237 


<209 


209 


222 


235 


<210 


210 


221 


231 


Mathematics 


<213 


213 


232 


243 


<212 


212 


232 


243 


<215 


215 


230 


240 




Grade 7 




Linear Regression 


Second-order Regression 


Rasch Status-on-Standard 




BB 


B 


P 


A 


BB 


B 


P 


A 


BB 


B 


P 


A 


Reading 


<208 


208 


226 


245 


<208 


208 


226 


242 


<210 


210 


226 


238 


Language Usage 


<209 


209 


225 


241 


<207 


207 


223 


237 


<211 


211 


224 


235 


Mathematics 


<220 


220 


238 


249 


<223 


223 


241 


250 


<223 


223 


238 


247 







Grade 8 




Linear Regression 


Second-order Regression 


Rasch Status-on-Standard 




BB 


B 


P 


A 


BB 


B 


P 


A 


BB 


B 


P 


A 


Reading 


<211 


211 


230 


248 


<210 


210 


229 


244 


<213 


213 


229 


240 


Language Usage 


<211 


211 


228 


245 


<209 


209 


227 


240 


<213 


213 


227 


235 


Mathematics 


<227 


227 


248 


259 


<224 


224 


247 


256 


<228 


228 


247 


256 



Establishing RIT score estimates for PACT performance levels. 

Once the cut scores were estimated from the three methods, we evaluated each set of possible cut scores 
to determine how accurately it predicted students’ actual performance on the corresponding PACT 
assessment. The most accurate method of prediction was generally used to derive the best estimate of 
RIT cut scores that equate to the different PACT performance levels. 

The following methods were used to establish the most accurate method for each performance level: 

• Below Basic and Basic. We selected the method that correctly identified the largest portion of 
students who scored in the below basic category on PACT. 

• Proficient. We calculated a prediction index statistic, which is one minus the ratio of Type I 
errors to correct predictions that is based on the proposed proficient cut score and selected the 
method that produced the highest result. 

• Advanced. We selected the method that correctly identified the largest proportion of students 
who scored in the advanced category on the PACT. 

Tables 4, 5, and 6 show the recommended RIT cut scores for each of the PACT performance levels. In 
general, Rasch SOS methods were most reliable for establishing predictive cut scores for the highest and 
lowest performance levels, while linear and second order regression proved more effective for estimating 
cut scores at the proficient level. 

In terms of predicting proficiency status, the best RIT cut score estimates for each grade correctly 
predicted PACT proficiency status for 79% to 87% of the cases in mathematics, for 80% to 83% of the 
cases when using language usgage to predict PACT ELA, and for 80% to 82% of the cases when using 
reading to predict PACT ELA. In terms of predicting performance level, the best RIT cut score estimates 
correctly assigned performance levels for 60% to 68% of cases in mathematics, 59% to 67% of cases using 
language usage , and 59% to 67% of cases using reading. 





Table 4 - Recommended RIT cut scores for PACT ELA performance levels - Reading 





Below Basic 




Basic 


Proficient 






Advanced 






Grade 


Score 


Method 


%of 

students 

ID 


Score 


Score 


Method 


Perf. 

Index 


Score 


Method 


%of 

students 

ID 


3 


<182 


R 


52.9% 


182 


196 


SR 


.887 


211 


R 


57.8% 


4 


<194 


R 


55.3% 


194 


209 


LR 


.881 


226 


R 


37.7% 


5 


<202 


R 


57.9% 


202 


218 


L 


.901 


232 


R 


27.1% 


6 


<210 


R 


68.7% 


210 


222 


L 


.899 


233 


R 


39.9% 


7 


<210 


R 


58.3% 


210 


226 


LSR 


.885 


238 


R 


37.4% 


8 


<213 


R 


58.4% 


213 


230 


L 


.898 


240 


R 


44.0% 



(L= Linear Regression, S=Second Order Regression, R=Rasch SOS method) 

Table 5 - Recommended RIT cut scores for PACT ELA performance levels - Language Usage 





Below Basic 




Basic 


Proficient 






Advanced 






Grade 


Score 


Method 


%of 

students 

ID 


Score 


Score 


Method 


Perf. 

Index 


Score 


Method 


%of 

students 

ID 


3 


<186 


R 


53.0% 


186 


198 


SR 


.894 


213 


R 


65.1% 


4 


<197 


R 


57.1% 


197 


211 


L 


.884 


226 


R 


50.5% 


5 


<204 


R 


57.6% 


204 


221 


S 


.934 


231 


R 


41.9% 


6 


<210 


R 


59.1% 


210 


222 


LS 


.910 


231 


R 


45.2% 


7 


<211 


R 


62.7% 


211 


225 


L 


.898 


235 


R 


41.0% 


8 


<213 


R 


55.4% 


213 


228 


L 


.895 


237 


R 


50.0% 



Table 6 - Recommended RIT cut scores for PACT performance levels - Mathematics 





Below Basic 




Basic 


Proficient 






Advanced 






Grade 


Score 


Method 


%of 

students 

ID 


Score 


Score 


Method 


Perf. 

Index 


Score 


Method 


%of 

students 

ID 


3 


<193 


R 


46.6% 


193 


212 


L 


.908 


220 


R 


55.2% 


4 


<202 


R 


60.5% 


202 


219 


LR 


.899 


228 


R 


79.8% 


5 


<212 


R 


62.6% 


212 


229 


S 


.905 


236 


R 


73.8% 


6 


<215 


R 


60.8% 


215 


232 


LS 


.925 


240 


R 


78.4% 


7 


<223 


SR 


67.0% 


223 


238 


LSR 


.905 


247 


R 


74.3% 


8 


<228 


R 


67.9% 


228 


248 


L 


.938 


256 


R 


67.9% 










We evaluate the relative accuracy of state alignment studies by comparing the prediction index statistics 
generated by these studies for accuracy in assessing proficiency status and performance level. Table 6 
summarizes the accuracy of proficiency status prediction for this study relative to other state alignment 
studies and Table 7 summarizes the accuracy of performance level prediction. The results show that the 
prediction index statistics for proficiency status prediction are low when compared to other state studies. 
In addition, prediction index statistics for this study are slightly lower than those generated by our study 
of 2002 PACT data. 

In terms of prediction of performance level, the prediction index statistics were in the lower half relative 
to those generated from other state studies and slightly lower than those generated by our prior study. In 
the case of reading and language usage, one reason for the comparatively low prediction index scores may 
have to do with content alignment. Since the PACT English/Language Arts assessment combines 
reading, language usage, and writing performance, it tests elements that will not be covered by a reading 
or language usage test that stands alone. 



Table 6 — Prediction Indices (Based on Proficiency Status) for Previous NWEA State Alignment 
Studies 



State 


Reading 


State 


Language 


State 


Math 


Texas 


.974 


Texas 


.968 


Texas 


.970 


Washington 


.971 


South Carolina Exit 


.938 


Wyoming 


.961 


Minnesota 


.944 


California 


.913 


Colorado ‘01 


.957 


South Carolina 
Exit 


.940 


Indiana ‘01 


.907 


Washington 


.949 


Wyoming 


.931 


Colorado ‘03 


.903 


Illinois 


.946 


Colorado ‘03 


.931 


Indiana ‘03 


.894 


Colorado ‘03 


.943 


Illinois 


.928 


South Carolina ‘04 


.889 


South Carolina ‘03 


.943 


California 


.925 


Arizona 


.874 


Minnesota 


.936 


Arizona 


.912 






Washington 


.936 


Colorado ‘01 


.910 






South Carolina Exit 


.933 


Nevada 


.902 






Arizona 


.919 


South Carolina ‘03 


.902 






South Carolina ‘04 


.914 


Indiana ‘01 


.902 






California 


.910 


Indiana ‘03 


.900 






Indiana ‘01 


.899 


Washington 


.886 






Nevada 


.866 


South Carolina ‘04 


.884 






Indiana ‘03 


.860 




Table 7 - Prediction index scores by performance level assignment for previous NWEA state 
alignment Studies 



State 


Reading 


State 


Math 


Washington 


.874 


Washington 


.928 


Texas 


.868 


Texas 


.900 


Indiana 


.860 


Illinois 


.888 


Colorado 


.840 


Colorado 


.808 


Illinois 


.804 


Washington 


.805 


Nevada 


.776 


Indiana 


.804 


South Carolina ‘03 


.757 


South Carolina ‘03 


.764 


Arizona 


.756 


Arizona 


.756 


South Carolina ‘04 


.717 


Nevada 


.742 


Washington 


.698 


South Carolina ‘04 


.741 


South Carolina Exit 


.649 


South Carolina Exit 


.705 


Minnesota 


.627 


Minnesota 


.611 


California 


.600 


California 


.565 



Figures 1 and 2 seem to illustrate the possible problem with content alignment between the Reading and 
Language Usage assessments and the PACT ELA test which combines the two domains. Figure 1 shows 
a scatterplot of Reading RIT against the PACT ELA scores for grade 6, the grade at which the predictive 
effectiveness of NWEA RIT scores was lowest. The red lines on the chart depict the predicted RIT cut 
score for Basic performance relative to the actual cut score on the PACT. There is a range of over 80 
PACT score points around the projected RIT cut score. Figure 2 shows a scatterplot of Mathematics RIT 
scores against the PACT mathematics scores for grade 6, the grade at which the predictive effectiveness 
of NWEA RIT scores was highest. Note that the range of PACT scores around the predicted Basic is 
considerably smaller, about 60 PACT score points, than the range for grade 6 Reading. Figure 1 shows 
considerably greater dispersion in scores than Figure 2. 

The illustration shows that the PACT’S inclusion of Reading, Language Usage, and writing elements in a 
single assessment probably makes it more difficult for an assessment in one domain to predict proficiency 
status or performance levels on the PACT with the level of accuracy that NWEA is able to achieve when 
the state assesses these domains individually. Despite this, the accuracy of PACT prediction from RIT 
scores, which is consistently above 80% when assigning proficiency status, and consistently above 60% 
when assigning performance level, is high enough to be useful as a tool to help teachers identify students’ 
probable status relative to these standards. 




Using RIT scores to estimate student probability of achieving passing 
performance on the PACT 

Tables 8, 9, and lOshow the proportion of students at each 5 point RIT level who earned scores at or 
above the Proficient Level on their respective PACT assessment. Using reading as an example, we find 
that about 24% of the Grade 5 students who achieved a reading RIT score between 210 and 214 went on 
to achieve a passing score on the PACT ELA assessment. A reading teacher would know that only about 
one in four of these students will be proficient on the PACT unless they work harder, receive more 
focused instruction, or have access to additional resources. 

On the other hand, about 95% of students performing between 230 and 234 achieved proficiency on the 
South Carolina ELA assessment. Teachers should feel free to focus their efforts with these students on 
content and skills that go beyond the minimum expectations for performance. 

Eigures 3, 4, and 5 are graphic depictions of the data in the tables. 

Table 8 - Proportion of students passing the PACT reading based on same spring RIT reading 
score 



RIT 


Grade 3 


Grade 4 


Grade 5 


Grade 6 


Grade 7 


Grade 8 


150 


0.00% 












155 


9.09% 












160 


1 0.53% 












165 


3.57% 












170 


8.16% 












175 


1 0.58% 












180 


1 6.34% 


6.78% 










185 


22.02% 


0.95% 


0.00% 


0.00% 






190 


33.78% 


3.64% 


1 .02% 


1 .64% 






195 


62.03% 


7.16% 


1 .55% 


1 .43% 


0.00% 


0.00% 


200 


81.04% 


19.18% 


3.40% 


2.11% 


2.11% 


0.93% 


205 


92.18% 


44.62% 


1 0.94% 


2.71% 


0.95% 


1 .06% 


210 


98.19% 


67.83% 


24.24% 


1 3.02% 


4.68% 


1 .23% 


215 


99.04% 


86.88% 


48.77% 


32.03% 


14.18% 


7.14% 


220 


1 00.00% 


95.42% 


73.51% 


56.51% 


29.81% 


16.81% 


225 




96.50% 


88.40% 


74.35% 


53.51% 


38.17% 


230 




97.62% 


95.42% 


85.30% 


73.45% 


61.69% 


235 




1 00.00% 


97.92% 


97.20% 


90.20% 


83.29% 


240 






1 00.00% 


1 00.00% 


97.70% 


95.57% 


245 










1 00.00% 


95.00% 


250 












90.00% 


255 












1 00.00% 





Table 9 - Proportion of students passing the PACT ELA based on same spring RIT language usage 
score 



RIT 


Grade 3 


Grade 4 


Grade 5 


Grade 6 


Grade 7 


Grade 8 


165 


0.00% 












170 


8.00% 












175 


2.78% 












180 


1 0.39% 


0.00% 


0.00% 








185 


8.51% 


3.45% 


4.00% 








190 


1 5.87% 


4.46% 


4.88% 


0.00% 






195 


21.22% 


7.94% 


1 .98% 


4.41% 


0.00% 


0.00% 


200 


36.09% 


1 6.43% 


6.91% 


8.61% 


7.84% 


1 .79% 


205 


50.62% 


35.82% 


1 7.42% 


16.41% 


5.85% 


0.86% 


210 


70.28% 


59.07% 


31.34% 


25.23% 


9.29% 


2.24% 


215 


83.06% 


76.62% 


51.79% 


47.29% 


23.98% 


9.39% 


220 


90.08% 


90.08% 


76.23% 


71.08% 


46.67% 


18.10% 


225 


90.00% 


95.28% 


92.78% 


89.81% 


74.64% 


39.63% 


230 


1 00.00% 


97.44% 


97.48% 


98.00% 


88.13% 


65.90% 


235 




1 00.00% 


1 00.00% 


1 00.00% 


97.87% 


83.97% 


240 










1 00.00% 


97.73% 


245 












1 00.00% 





Table 10 - Proportion of students passing the PACT mathematics based on same spring RIT 
mathematics score 



RIT 


Grade 3 


Grade 4 


Grade 5 


Grade 6 


Grade 7 


Grade 8 


180 














185 


0.91% 












190 


4.24% 












195 


6.24% 


0.00% 




0.00% 






200 


17.61% 


2.93% 


0.00% 


1 .05% 






205 


35.34% 


1 0.59% 


1.52% 


1 .83% 


0.00% 




210 


56.40% 


1 5.66% 


2.68% 


2.18% 


1 .57% 




215 


74.36% 


40.23% 


6.79% 


5.16% 


0.45% 


0.00% 


220 


87.40% 


65.71% 


20.70% 


1 4.22% 


2.26% 


1 .27% 


225 


95.60% 


87.06% 


44.04% 


31.82% 


5.20% 


0.32% 


230 


97.37% 


96.04% 


71.91% 


59.40% 


1 7.94% 


1.41% 


235 


1 00.00% 


97.86% 


91.48% 


79.10% 


46.58% 


6.27% 


240 




1 00.00% 


96.23% 


95.73% 


69.39% 


21.17% 


245 






99.31% 


98.87% 


89.66% 


48.27% 


250 






1 00.00% 


1 00.00% 


97.72% 


74.94% 


255 










1 00.00% 


91.17% 


260 












1 00.00% 
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Figure 3 - 



Percent of Students Achieving Proficient Score on South Caroiina PACT 
Engiish/Language Arts - NWEA Reading 




Reading RIT Score 
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Percent of Students Achieving Proficient Score on South Caroiina PACT 
Language Usage 




Figure 5 - 



Percent of Students Achieving Proficient Score on South Caroiina PACT 

Mathematics 




Comparing South Carolina PACT standards with the estimated standards 
reported in other state test alignment studies 

Northwest Evaluation Association tests have been aligned with the cut scores for the state high school 
standards and/or proficiency tests in eight states. To get an estimate of the difficulty of the PACT in 
relation to other state tests, we evaluated the standard defined as the NCLB passing score and compared it 
to the cut score representing the same standard in these other states. 

The results are summarized in Table 11. South Carolina’s cut scores in reading are lower than five of the 
eight states studied. The cut scores in mathematics are the lowest of any state studied. We’d recommend 
caution about drawing any judgments about the quality of South Carolina’s standards from that 
information. States establish standards for different purposes. States also attach different stakes to their 
standards. Some states, Oregon might be an example, set their high school standards prior to the adoption 
of NCLB. In Oregon’s case, these standards were set at a level they believe appropriate for students 
pursuing some form of post-secondary education. In addition, Oregon does not require that students pass 
these standards as a condition for graduation. This confluence of factors explain why the Oregon 
standard was set relatively high. 

Other states, California would be an example, established high school performance standards after the 
passage of NCLB. They were intended to reflect performance needed to pursue post-secondary 
education. They were however intended to be a prerequisite for graduation, although the state has 
postponed the requirement for now. Given that the standards were implemented with the intention that all 
students would be required to achieve this level of performance, it is not a surprise that the California 
standard is not as rigorous as Oregon’s. 




In general, standards should be judged on how well they align with the purposes the community has set 
for establishing standards, not purely on how high or low the “bar” is set. One thing the tables make clear 
is that proficiency standards vary widely from state to state and that proficiency is not yet a concept that 
has a shared definition, although greater consensus in standard setting seems to be emerging. It would be 
fair to say that most states that we have studied who have set standards since implementation of No Child 
Left Behind has begun have tended to establish standards near or below the 50* percentile on our norms. 

South Carolina implemented the PACT to meet the No Child Left Behind Act’s requirements for high 
school assessment. Passing the test is a prerequisite for graduation, so the stakes associated with this test 
are high. These factors may explain, at least in part, why the standard is low relative to some other states. 




Table 17 - Cut scores representing “proficient” or “meets standards” level of performance on 16 state assessments 
Reading 



Grade 3 


Grade 4 


Grade 5 


Grade 6 


Grade 7 


Grade 8 


Grade 9 


Grade 1 0 


State 


Cut 

Score 


%ile 


State 


Cut 

Score 


%ile 


State 


Cut 

Score 


%ile 


State 


Cut 

Score 


%ile 


State 


Cut 

Score 


%ile 


State 


Cut 

Score 


%ile 


State 


Cut 

Score 


%ile 


State 


Cut 

Score 


%il 

e 


SC03 


205 


67 


WY 


214 


73 


SC03 


220 


73 


SC04 


222 


64 


SC03 


227 


70 


WY 


232 


74 


MT 


224 


43 


OR 


236 


77 


NV 


202 


58 


SC03 


213 


70 


SC04 


218 


68 


SC03 


221 


63 


WA 


226 


67 


SC03 


230 


68 


lA 


224 


43 


WA 


227 


53 


CA 


200 


51 


SC04 


209 


59 


NV 


215 


59 


CA 


216 


46 


SC04 


226 


67 


SC04 


230 


68 


ID 


221 


37 


ID 


224 


44 


SC04 


196 


42 


WA 


207 


53 


CA 


214 


56 


MT 


211 


35 


CA 


221 


50 


OR 


227 


58 


CO 


204 


9 


MT 


224 


44 


OR 


193 


35 


CA 


205 


46 


PA 


212 


50 


ID 


211 


35 


MT 


218 


43 


CA 


226 


54 








SCL3 


224 


44 


ID 


193 


35 


ID 


200 


34 


AZ 


210 


45 


IN 


210 


32 


lA 


216 


37 


AZ 


224 


49 








lA 


223 


42 


MT 


193 


35 


MT 


196 


26 


OR 


209 


42 


lA 


209 


30 


NV 


215 


35 


PA 


223 


46 








CO 


209 


15 


IL 


193 


35 


lA 


196 


26 


IL 


207 


37 


TX 


208 


28 


ID 


215 


35 


IN 


219 


35 








SCL2 


209 


15 


IN 


192 


32 


NV 


194 


22 


MT 


206 


35 


CO 


197 


11 


TX 


210 


24 


MT 


219 


35 








CA 


208 


14 


lA 


191 


31 


CO 


191 


18 


ID 


206 


35 








CO 


206 


18 


lA 


219 


35 














AZ 


190 


29 








lA 


205 


32 














ID 


218 


32 














TX 


179 


13 








TX 


204 


30 














IL 


218 


32 














CO 


179 


13 








CO 


197 


18 














MN 


218 


32 












































CO 


206 


12 















For grade 1 0, SCL2 represents the minimum score required to pass the HSAP while SCL3 represents the score reflecting “proficient” performance 
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Table 1 8 - Cut scores representing “proficient” or “meets standards” level of performance on 1 6 state assessments - 
Mathematics 



Grade 3 


Grade 4 


Grade 5 


Grade 6 


Grade 7 


Grade 8 


Grade 9 


Grade 1 0 


State 


Cut 

Score 


%ile 


State 


Cut 

Score 


%ile 


State 


Cut 

Score 


%ile 


State 


Cut 

Score 


%ile 


State 


Cut 

Score 


%ile 


State 


Cut 

Score 


%ile 


State 


Cut 

Score 


%ile 


State 


Cut 

Score 


%ile 


SC04 


212 


84 


WY 


221 


83 


SC04 


230 


81 


SC03 


235 


78 


SC03 


242 


78 


WY 


257 


89 


MT 


242 


47 


WA 


257 


73 


SC03 


208 


75 


SC04 


219 


78 


SC03 


227 


76 


SC04 


232 


73 


WA 


242 


78 


SC03 


251 


80 


lA 


241 


44 


MT 


247 


40 


CA 


204 


63 


WA 


218 


76 


CA 


225 


71 


CA 


230 


68 


SC04 


241 


76 


AZ 


248 


75 


ID 


240 


42 


lA 


247 


40 


NV 


203 


59 


SC03 


217 


74 


AZ 


220 


59 


IN 


221 


47 


CA 


238 


71 


SC04 


247 


73 


CO 


235 


32 


OR 


245 


33 


IN 


201 


50 


CA 


212 


59 


NV 


216 


48 


ID 


219 


42 


ID 


225 


44 


CA 


240 


60 








ID 


242 


25 


OR 


199 


46 


ID 


205 


39 


PA 


216 


48 


lA 


218 


40 


MT 


224 


42 


PA 


237 


53 








SCL3 


237 


18 


AZ 


199 


46 


lA 


205 


39 


OR 


215 


46 


MT 


218 


40 


lA 


222 


38 


OR 


235 


50 








CO 


233 


14 


MT 


197 


39 


MT 


205 


39 


ID 


213 


41 


CO 


207 


19 


TX 


221 


35 


ID 


233 


46 








CA 


232 


13 


lA 


197 


39 


NV 


200 


26 


MT 


212 


38 








NV 


220 


33 


MN 


231 


42 








SCL2 


223 


7 


ID 


196 


36 








lA 


212 


38 








CO 


216 


26 


IN 


231 


42 














IL 


193 


29 








IL 


210 


33 














IL 


230 


40 


























TX 


209 


31 














MT 


228 


36 


























CO 


201 


15 














lA 


228 


36 












































CO 


225 


31 
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Summary and Conclusions 

This study investigated the relationship between the scales used for the PACT assessments and the RIT 
scales used to report performance on Northwest Evaluation Association tests. The study determined the 
reading, language usage and mathematics RIT score equivalents for the PACT performance levels in 
English/Language Arts and Mathematics. Test records for more than 22,000 students were included in 
this study. 

Three methods generated an estimate of RIT cut scores that could be used to project PACT performance 
levels. Rasch SOS methods generally produced the most accurate cut score estimates. Accuracy of 
predicting PACT passing performance was above 80% for nearly all grades and subjects when using the 
best methodology. 

Readers should exercise some caution about generalizing these results to their own settings. Curricular or 
instructional differences unique to your districts may influence the accuracy with which the estimated cut 
scores reflect actual performance in your setting. With this limitation in mind, we would encourage 
educators to use this data as one tool to inform standards-based decisions. 

The information gathered in this study came from measures employing the NWEA RIT Scale. Because 
all of the research that we have to date indicates that scores generated from computer-based tests and 
Achievement Level Test (ALT) scores are virtually interchangeable, readers should feel comfortable 
applying the results of this study in any setting that uses the RIT scale. 

We hope that data from this study provides useful information to help South Carolina educators use 
NWEA assessments to better inform, plan and deliver student instruction. Good information, when 
matched with the professionalism and commitment of our South Carolina colleagues, will assure that 
every student has the opportunity to reach their aspirations. 
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